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ABSTBACT 

An expanded duties role of the «ultiple*progra« 
evaluator as an integral part of the ongoing decision^iaking process 
in all projects served is defended. Assuiptions discussed included 
that need for projects with related objectives to pool resources and 
avoid duplication of effort and the evaluator 's unique ability to 
provide an objective cosnunication conduit among such projects* 
adainistrative and field personnel. It is observed that evaluators 
already perform this function unofficially in Many cases. The study 
concludes that such a function should be recognized as a legitisate 
and important part of the evaluator* s role. (Author/RC) 
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Setting the Scene ^ 

As a research' associate for the School District of Philadelphia, the 
author's job Is to evaluate six to twelve federally funded projects In career and 
vocational education. Each of these projects Is centrally administered, and may 
involve any number of schools between one and twenty-five, while any given school 
may be involved with as many as three or four such projects simultaneously. Pro- 
^ect objectives relate predominantly to fostering career awareness and occupational 
skill development in students. While evalution activities ava geared to both pro- 
cess and outcome objectives, this paper will present some observations and view- 
points related primarily to process considerations. 

The author defends an expanded duties role of the multiple-project evaluator 
as both an interproject coordinator and an integral part of the on-going decision 
making process, in addition to his/her role as evaluator, for all projects served. 
While this role is not necessarily recommended for all educational evaluator s, it 
is seen as an accomplished, though officially unrecognized, fact for those serving 
a multiplicity of functionally interrelated projects within one organization. 

The Case for an Expanded Duties Role 

By assuming an external, objective stance, the evaluator is often better 
able to observe project strengths, weaknesses, and duplication of effort than are 
project personnel with greater ego involvement in a project's success. It is 
therefore felt that the evaluator can best provide an objective communlcatiop con- 
duit among the administrative and field personnel of projects whose similar or 
complementary objectives might better be achieved through either a coordination 
of effor'J or a refocusing of energy. 

To this end, the Division of Priority Operations Evaluation Services of the 
Office of Research and Evaluation, School District of Philadelphia, has in- 

O 

stituted an ongoing system of data feedback to project administrators concerning 
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both process and outcome issues. While this has led to a continuing examination 
of project objectives, methods, and achievements, informal evaluator feedback 
and input have often led to more immediate program revisions than are possible 
through existing formal channels. A sample of such interventions, for both in- 
dividual and related projects, follows. (For further information see Chern et al, 
1972,73,74.) 

The author evaluated a project which provides small group career guidance to 
ninth-grade students. One group, seated in a circle, was observed participating 
in a variety of verbal activities for approximately 90 minutes. After 50 minutes > 
several students began to fidget, talk among themselves, and leave their seats. 
In a feedback session iimnediately following the activity, the evaluator, who ini- 
tiated the feedback session, related this observation to the field personnel who 
revealed that they, too, had observed a waning of interest in the students, but 
were unprepared to deal with this except by cajoling students to continue in their 
participation. Alternatives were discussed, and it wa?3 agreed that activities 
involving physical movement would be incorporated into the counseling program, on 
the assumption that both the physical activity and a change of pace would help 
maintain student interest. Subsequent observations revealed that such activities 
were being used to a certain extent and were helping to keep interest high. 

A second project evaluated by the author provides high school students with 
work experiences in hospital labs an^l clinics. The program is divided into a 
summer phase, during which students are paid a stipend for full-time work, and a 
school year phase, during which students work part time without remuneration. 
Most students remain in the program for the full year, working with the same 
hospital staff member, or preceptor. In the four years of th3 project's existence, 
there has generally been a lag of several months between the close of the summer 
phase and the start of the school year phase. This has been due to th^ time re- 
qjuired to recruit new participants, staff vacations, and faulty communications 



between school and hospital personnel. In 1974-75, the evaluator served un- 
officially as the project coordinator, contacting preceptors to determine whether 
their summer assistants had returned to work by October. They had not. He 
than informed the high school coordinator that students had not returned to their 
labs» a3 'hough the preceptors were awaiting their return. By November, the old 
students had begun returning to "work," although a new group (to replace those 
who had graduated) had not yet been selected. 

Without these two interventions by the evaluator, the projects would pro- 
bably have continued to function on a less than optimal level. 

The third example involves three projects emanating from three different de- 
partments within the system; all federally funded, with complementary if not 
similar objectives, and all located at the same school. Yet without the evaluator 's 
unofficial intervention, none of these projects would have touched base with the 
other two in a manner which would allow each to serve as a resource for the others. 

The first project involved 12 of the school's faculty members in weekly staff 
development sessions ia which they were shown, and in %rtiich they discussed, methods 
of incorporating career decision making concepts and activities into their ongoing 
teaching and counseling programs. 

The second project involved one teacher (also involved In the first project) 
in an out-of-school staff and curriculum development effort with other teachers 
iT» the same field. Its purpose was to revise the ninth grade Industrial Arts 
curriculum in such a manner as to emphasize the career development aspect of tKi^T 
area of instruction, incorporating elements of all academic disciplines as welL. 

The third project brought a combined hands-on and group counseling ex- 
perience to the school's ninth grade students for three weeks. Its purposes 
were to foster greater career awareness and self awareness in relation to c.iireers, 
in order to help students select a high school curriculum leading toward entry 



into a career area of their choosing. (An instance of the group counseling 
phase of this project has been cited above.) 

After obscirviag the staff development and curriculum development projects 
for several months without observing any interaction between these two projects 
at the school, the evaluator asked the staff development participants how they 
had incorporated career decision making concepts and methods into their ongoing 
programs. Each participant enumerated ways in which this was being accomplished 
wichin the framework of his/her own area, except for the teacher participating 
in th** curriculum development project. According to him, he was too busy working 
on the new curriculum to implement any aspect of career decision making in his 
classroom. (He was, however, being paid to attend the staff development sessions.) 
Ironically, career decision making was an integral concept in the new Industrial 
Arts curriculum. But neither this teacher nor the rest of the starff development 
group grasped the interrelatedness of these two projects, so that no attempt was 
made to use or4e as a resource or complement for the other • 

Immediately at the evaluate:: 's suggestion, the staff development group ex- 
plored ways in which all could relate their own curricula to the Industrial Arts 
curriculum. Many parallels were uncovered in most of the subject areas represented, 
including English, Math, Social Studies, Science, Art, and counseling. Several 
participants subsequently observed activities in the Industrial Arts classroom, 
but no formal attempt was made to coordinate the two projects beyond the steps 
which have been described. 

In the final example to be presented, the third project described (hands-on 
experiences combined with group career guidance) visited this school for a three- 
week period, addressing itself to all ninth grade students. The school served 
students in grades seven through nine; the staff development sessions involved 



faculty members who worked with students in all three grades; and the Industrial 
Arts project was geared prima^-ily to the ninth grade curriculum. 

During a chat with the field supervisor of the hands-on/guidance project, 
before It visited this school, the evaluator asked whether any plans had been 
made to prepare the faculty for the project's arrival. It was learned that a re- 
quest had been made to the principal to conduct a faculty orientation, and that 
project personnel were granted only a few minutes to accomplish this purpose 
during a weekly faculty meeting. The author then asked whether the supervisor 
had contacted the coordinator of the staff development project in an effort to 
have at least one session devoted to an orientation, as well as to assess the 
attitudes and knowledge of the staff development participants regarding career 
development issues in general and the hands-on/guidance project in particular. The 
supervisor replied that one informal communication had occurred, but that it had 
not been followed up. The supervisor agreed that such an arrangement might prove 
beneficial, but did not feel that she should be the one to initiate a contact 
with the staff development coordinator. 

A similar communication took place between the evaluator and the staff de- 
velopment coordinator, revealing a similar reluctance on her part to initiate 
a contact with the supervisor. The writer intervened by personally phoning the 
coordinator from the supervisor's office, explaining to both that he felt it 
would be in the best interest of both programs, and hence the students served by 
both programs, if a cooperative venture were undertaken. (The representative of 
the curriculum project was also included in this joint venture by virtue of his 
participation in the staff development project.) A three-way phone discussion 
ensued, during which an arrangement was made for the hands-on/guidance staff to 
meet with the staff development group during a regularly-scheduled two-hour 
session.. 
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During the joint session, numerous ways were explored in which all three 
projects represented could cooperate to achieve the common objectives of helping 
students gain career awareness and awareness of themselves in relation to careers. 
Subsequent observation revealed that such interproject cooperation ended as 
rapidly as ic had begun. 

Some Issues 

It is hoped that the preceding descriptions of evalitator interventions have 
raised a number of issues in the reader's thoughts. Among these, the author 
considers the following important. 

First, the project evaluator is probably in a better position to objectively 
assess the presence or absence of, or need ^f or, articulation, of efforts, among pro- 
jects with related goals. 

Second, we must consider whether it is within the evaluator *s jurisdictiou 
to intervene directly into the ongoing activities of projects, either to faci- 
litate the achievement of objectives or to coordinate efforts of a number of pro- 
jects. 

Third, we must consider whether, and under what circumstances, the evaluator 
can maintain objectivity while intervening in the day-to-day operations of projects 
being evaluated. 

Fourth, at least two levels of process evaluation are implied. That de- 
scribed in the present paper places the evaluator in direct, ongoing interaction 
with project field personnel. The other level views the project evaluator as 
providing information to project personnel which must pass through the hands of 
the evaluator*s superior(s) and the project adiAinistrator(s) before it reaches 
the field personnel of the projects being evaluated. The simple "tinker-toy" 
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diagram In Figure 1 can be used to Illustrate the number and types of channels 
It Is possible for evaluation Information to take, from field observations back 
to field implementation. It becomes apparent that the less bureaucratlzed the 
communication pattern, the faster Important evaluative information can be trans- 
mitted to the field, where it might lead to needed revision in implementation pro- 
cedures. 
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Figure 1. Diagram of existing communications channels between evaluation and 
project personnel. 
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Some Alternatives 

The crux of the problem, as this author sees it, is the slowness in official 
communication of evaluation data which could prove instrumental in helping pro- 
jects to become more effective, coupled with an unrealistically narrow conception 
of project evaluation in a large, multifaceted institutional setting. 

In ordf.r to facilitate the flow of information from field observations back 
to field implementation, and thereby help programs make needed revisions in a 
harry, this paper proposes that new roles and responsibilities be explored for 
^valuators working in large public school systems. 

Evaluators serving any number of projects might have immediate and direct 
access to all administrators of those projects at all administrative levels, for 
the express purpose of feeding back infcrnation which may necessitate immediate 
project revision. In cases where an evaluatcr is serving several functionally 
related or complementary projects operating under different hierarchies or 
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branches of the same hierarchy, the evaluation strategy might be designed so 
that a given time slot is reserved at regular intervals (e.g. weekly) during 
which the evaluator reports to key decision makers for all involved projects 
simultaneously. These reporting sessions could result in instant recomir^.ndations 
regarding needed revisions in schools or other locations in which the projects 
function. 

A second evaluation strategy, complementing the first, would require that 
all project designs be reviewed by a task force composed of evaluation and pro- 
ject personnel, before being submitted as proposals for funding. The purposes 
of .lis review would be to consolidate overlapping efforts, increase emphasis on 
those project components which have demonstrated greatest effectiveness, de-emphasize 
or revise those which have not, and develop contractual guidelines for the use 
of each project in the field. These guidelines, which would be agreed to and 
signed by project, field, and evaluation personnel would clearly spell out the 
obligations and responsibilities of all involved parties. For example, the 
evaluator would agree to evaluate specific objectives in specific ways and re- 
port findings by specific dates. (While this system is already in use by the 
Office of Research and Evaluation, it does not come into play until after projects 
have already received funding.) Project officials would agree to provide speci- 
fic resources to schools by specific dates, and principals would agree to pro- 
vide specific supportive services for each project located in their schools by 
their request. 

The two alternatives thus far proposed assume an interest and ability on the 
part of project and system officials to actively cooperate for the benefit of 
the students. Since political, financial, temporal, and other obstacles and pre- 
sures often impede inter-hierarchical cooperation, a third alternative is proposed 
in an attempt to transcend the constraints Imposed by the nature of the institution. 
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Ombudsmen or advocates could serve to promote optimal project utilization 
as well as th6 incorporation of projects demonstrated to" be effective into on- 
going school programs. One advocate would oversee each functionally-related 
group of projects designated as high priority by administration, such as basic 
skills development, career educat-.on, and early childhood education. Advocates 
would report to someone high enough in the administration, under whose authority 
they would function, that they could r.iove freely among all levels of all 
hierarchies administering related projects. It is viewed as essential that ad- 
vocates have a strong research background, for they would propose recommendations 
on the basis of evaluation findings, current research evidence, and their own 
observations. 

While it seems inefficient to create new positions in an already over- 
cluttered bureaucracy, these may be justified if they can serve as catalysts to 
promote more effective use of existing personnel and programs. The problem is 
that while sufficient resources already exist to implement necessary educational 
programs, the lack of communication and coordination among these resources 
prevents them from providing their services in a manner that would best facilitate 
learning. The alternatives proposed here seek to remedy the situation through 
extending the working definition of institutional research and evaluation to 
include developmental, planning, and intervention functions. 
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